METHOD AND APPARATUS FOR VIDEO COMPRESSION AND 

RESTRUCTURING 

5 riELD OF THE INVENTION 

The present invention generally relates to a method and apparatus for 
\ideo compression and reformatting, and particularly to a method and 
apparatus for enabling existing video channels to accommodate the 
transmission of more video programs. 

10 

BACKGROUND OF THE IN^NTION 

It is getting more and more important to tackle the problem that the 
signal flow capacity and communication quality of a video transmission 
systems are always limited by channel bandwidth. This is because of the bulk 

15 of daia contained in video signal and the higher communication quality 
recjiiired by video transmission. Although the number of video channels 
:^i!ov. able for cable TV is more than 120, only very few remain available given 
that some are prohibited or are not suitable for using, resulting in extreme 
ditficulty for acquiring channels for new video programs and new medium 

20 broadcast companies. Due to the limited number of channels available for 
application, the increase of number of tlie programs in a single channel shall be 
a good option for expanding the number of broadcast programs which, however, 
is limited by channel bandwidth, leading to a necessity of digital signal 
compression technology. 

25 Shown in Fig. 1 is a conventional digital video signal transmission and 

receiving system in which 8 video programs are merged by multiplexing 
teclinoiog>^ into a single video channel for transmission. As shown in Fig. 1, 
the system comprises a sending station KX) for sending processed video 
program^s which are then transmitted tlirough cable 200 to a receiving site to be 

30 received by STB (set-top-box) 300 thereat and delivered therefrom to users. 
In sending station 100, a network management and control unit 110 
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is used to manage and control a subscriber management unit 112, a 
multiplex management unit 114, a conditicMial access unit 116, a multiplex 
and sever 118, and a 8:1 multiplexer 120;the input of 110 is connected to an 
electronic program guide 122 and a scheduler/trafficker 124 according to 
which a tape/archive 126 provides programs to be sent directly or through 
an encoder 128 to a multiplex and sever 118 which also receives a live 
video source 130 processed by a real-time encoder 132. Multiplex and sever 
118 in turn sends a video signal to be processed by the 8:1 multiplexer 120 
for merging into a single channel and then being processed by a modulator 
140 in order to transmit through cable 200. 

STB 300 receives, throu^ a cable interface 302, from cable 200 a 
signal which has frequency reduced by a tuner 304 and is then demodulated 
by a demodulator 306 into a MPEG2 video signal consisting of 8 programs, 
and is demuhiplexed afterwards by a 1:8 demultiplexer 308 into individual 
video signals for being applied to a bus 310 connecting a Direct Random 
Access Memory (DILAM) 312 and a flash memory 314 for saving data. 
Based on the system, the video programs selected by users are retrieved and 
saved in a DRAM 3 16 while the others are ignored. The signals contained 
in DR.AM 316 are decoded by MPEG2 decoder 318, with digital video 
signals (digital video data stream) and digital audio signals individually 
inputted to a video digital-to-analog converter 320 and an audio digital-to- 
analog converter 322 to be respectively converted into analog video signals 
and analog audio signals for outputting to an ordinary TV for displaying. 
The 1:8 demultiplexer 308 also connects a infrared ray receiver (IR) 324 
which is used by users to select a desired program through a remote 
controller. 

Given that the bandwidth of a channel for current TV systems is 
about 6Mhz v^ith transmission speed at about 27Mbps, and that a MPEG2 
system is adopted, the digital signals will usually be provided (by most of 
MPEG2 Encoder, for example) with an average output speed of 3.3Mbps. 
With 3.3Mbps X8=26.4Mbps<27Mbps (equation 1), it can be seen at 



aiost 8 programs can be accommodated in a channel, e.g., only 8 programs 
can be broadcast simultaneously through one channel even though a 
MPEG2 system is used, thereby the number of increased programs is far 
be\ end significant given that the number of available channels is so limited. 

Fig. 2 shows a video signal obtained from MPEG2 compression, 
most of vvliich are distributed in a small range of bandwidth, with scarce 
explosion 402 and swiftly moving rapid pan of high detail 404, implying 
feasible further compression. 

Fig. 3(A) and Fig. 3(B) illustrate encoding and decoding algorithm of 
MPEG2. As can be seen in Fig. 3(A), a MPEG encoder comprises a 
discrete cosine transform unit 502, a quantizer 504, and a variable length 
encoder 506, Usually a video signal is converted through the tliree devices 
into a bit stream (digital video data stream) to be sent to user sites through a 
modulator and transmission medium. To reduce the bulk of signal flow, 
nian\' frames in MPEG2 system are transmitted on the basis of the 
difference between two successive frames, therefore a MPEG2 encoder 
Panher comprises a motion compensation unit 512 and a motion estimation 
unit. Due to the need that tlie two devices must operate with video signal 
data, a dequantizer 516 and an inverse discrete cosine transform unit 5 18 are 
nuther required. The fmal output is a MPEG2 bit stream (digital video 
data stream). 

Fig. 3(B) illustrates the operation algorithm of a decoder, which 
re\-erses the operation shown in Fig, 3(A), i.e., the MPEG2 bit stream 
(digital video data stream) outputted by the encoder in Fig. 3(A) is 
inputted to the decoder in Fig. 3(B), and processed by a variable length 
decoder 522, a dequantizer 524, and an inverse discrete cosine transform 
unit 526, as well as a motion compensation unit 528, to eventually obtain a 
restored video signal as its output. 

When proceeding quantization, the bulk of video data signal may be 
reduced by lowering quantization level. Although lowered quantization 
level naturally reduces quantized data signal, it leads to a drawback that the 



quality of video frames is lowered. 

Paik suggested, in US patent 5,216,503, a multi-channel video 
compression system using a statistical multiplexer to integrate multiple 
video programs in a conventional video channel. To avoid the unnecessary 
5 waste resulting from too big instant bandwidth of a single program, a buffer 
controller is used to generate, when the total bandwidth of these programs 
exceeds system capacity, a signal for requesting the quantizer to adjust 
quantization level so that the bandwidth is lowered. 

When the aforementioned patent was filed, digital video signal 

10 standard had not been established, therefore its quantizer was designed for 
digitizing video signal (similar to MPEG). Nowadays some digital video 
signal standards such as ISO/TEC JTCI/SC29AVG11 for MPEG2 have 
been established, thereby most of the video contents are processed 
according to these standards, resulting in a necessity of converting digital 

15 video contents into analog contents if the aforementioned patent is to be 
applied, leading to the need of extra decoding devices and extremely long 
operating time. 

It can be seen now that a practicable method and apparatus for 
integrating multiple programs in a conventional video channel can be 
20 adopted only if it fits the existing video system and maintains the quality of 
video frames. The requirement, however, is beyond the capacity of 
conventional arts. 

SUMMARY OF THE INVENTION 

25 An object of the present invention is to provide a method and apparatus 

for integrating multiple video programs in a video channel. 

When bandwidth is extremely limited, digital video signal is further 
compressed according to the present invention under the condition that it is not 
to be sensed by the eyes of human being, leading to more efficient utilization of 

30 existing channels. It is therefore another object of the present invention to 
provide a method and apparatus for compressing and restructuring video 
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signals. 

.^oiother further object of the present invention is to promote operation 
efficiency of a video system by enabling a single channel to accommodate 
more video programs. 
5 The other further object of the present invention is to provide a method 

and apparatus for directly compressing video signals to realize a real time video 
system. 

Furthermore, digital video signals (digital video data stream) can be 
directly compressed according to the present invention to enable a single 
10 channel to accommodate more video programs, therefore it is also an object of 
the present invention to provide a video compressor and a method for 
compressing digital video data, as well as a trancoder and associated method 
for compressing digital video data. 

The trancoder suggested by the present invention is characterized in that 
15 u better quantization scale can be achieved by determining a new quantization 
scale when quantizing data. It is therefore also another object of the present 
in^'ention to provide a neural network quantization scale predictor for 
doiemiining an optimum quantization scale. 

Tlie compression of digital video signal suggested by the present 
20 in\'emion is characterized in that the quantization level for the areas of a 
video frame which are less sensitive to human eyes is reduced while the 
quantization level for those which are sensitive to human eyes is maintained 
the same. 

hi an embodiment of the present mvention, muhiple digital video 
25 compressing and restructuring devices (or called Q-mux) are used to 
directly compress digital video signals (digital video data stream) which are 
then integrated by a multiplexer; each digital video compressing and 
restructuring device has a multiplexer to restructure digital codes (digital 
codes of the multiple digital video signals) having been compressed by 
30 \'ideo compressors (or called Q-presser); each video compressor comprises 
at least a trancoder to reduce the quantization level for the areas of a frame 



which are less sensitive to human eyes, in order to further compress digital 
video signal. 

The present invention may best be understood throu^ the following 
description with refa-ence to the acconqjanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a conventional video transmission and receiving system 
wherein 8 video programs are merged in a single channel. 

Fig. 2 shows video signal bandwidth distribution of a conventional 
MPEG2. 

Fig. 3 shows an encoder and decoder of MPEG2, among which Fig. 
3(A) shows the encoder and Fig. 3(B) shorn the decoder. 

Fig. 4 illustrates, on the basis of video bandwidth distribution, the 
video compression algorithm suggested by the present invention. 

Fig. 5 illustrates an embodiment of the present invention allowing up 
to 24 video programs to be accommodated in a single channel. 

Fig. 6 illustrates an embodiment of a trancoder suggested by the 
present invention. 

Fig. 7 illustrates an embodiment of a quantization scale predictor 
suggested by the present invention, which is achieved by a neural network 
of 3 layers. 

Fig. 8 shows an embodiment of a video transmission and receiving 
system suggested by the present invention. 

Fig. 9 shows an embodiment of a video-on-demand analogy system 
(approximate to a video-on-demand system) suggested by the present 
invention. 
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MPEG2 decoder 
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video digital-to-analog converter 
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audio digital-to-analog converter 
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explosion 
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rapid pan of high detail 
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discrete cosine transform unit 




504 


quantizer 


30 


506 


variable length encoder 
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motion compensation unit 



516 dequantizer 

518 inverse discrete cosine transform unit 
522 variable length decoder 
524 dequantizer 

526 inverse discrete cosine transform unit 
528 motion compensation unit 

60 1 -608 digital video compressing and restructuring devices 

(or called Q-mux) 
611-613 video compressors 

621 trancoder 

622 input buffer 

623 output buffer 

624 disc drives (computer disc drives) 

625 high speed network 
631 multiplexer 

640 Ethernet network switch (etherswitch) 

650 8:1 multiplexe 

700 trancoder 

702 decoder 

704 encoder 

712 delay buffer 

714 quantization scale predictor 

716 variable length decoder 

718 dequantizer 

720 quantizer 

722 variable length encoder 

802 input layer 

804 concealed layer 

806 output layer 

90 1 -908 digital video compressing and restructuring devices 

910 multiplexer 
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set-top-box 
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digital video disc (DVD) 
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disc (hard or floppy) 


935 


cable system 
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satellite 


938-939 satellite antenna 


940 


head-end 



BL B2bit stream (digital video data stream) 
T^ETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

According to the present invention, digital video contents are directly 
Cvonipressed and multiple video programs are merged into a single video 
channel. It can be seen from Fig. 4 that most of video bandwidth ranges 
beioH 1Mbps, therefore further exploitation of bandwidth can be achieved 
b\' farther compressing digital video signals (digital video data stream). 

Fig. 5 shows an embodiment of the present invention, which 
comprises 8 digital video compressing and restructuring devices 601-608 
each including 3 video compressors, such as 3 video compressors 611- 
613 included in 60L each of video compressors 611-613 has a trancoder 
621 and buffers connected to its input and output. For example, video 
compressor 611 includes trancoder 621 for converting video codes of 
SJMbps or higher transmission speed into video codes of 1.1Mbps. 
Trancoder 621 has its input and output respectively connected to input 



buffer 622 and output buffer 623 each with a memory capacity of 1Mb for 
temporarily saving video signals. The digital video signals (digital video 
data stream) retrieved from disc drives 624 are compressed by trancoder 
621 to become video codes of 1.1Mbps. The video compressor may also 
receive digital video signals (digital video data stream) from another kinds 
of sources such as that video compressor 613 receives digital video signals 
(digital video data stream) from high speed network 625 and compresses the 
received digital video signals (digital video data stream). 

3 video compressor 611-613 output signals to be integrated by 
multiplexer 63 1 to form signals of 3.3Mbps. 8 digital video compressing 
and restructuring devices 601-608 output signals to be sent to 8: 1 multiplexe 
650 through etherswitch 640, and then integrated step by step to form digital 
video signals (digital video data stream) of 27Mbps to be outputted. 

Each video compressor in the embodiment compresses video signals 
into video codes of 1.1Mbps, each of digital video compressing and 
restructuring devices 601-608 has 3 video compressors and has output of 
3.3Mbps, outputs of 8 digital video compressing and restracturing devices 
601-608 fit right in a channel of 27Mbps, thereby a single channel can 
accommodate up to (3 X 8=)24 video programs which are 3 times what a 
conventional system can provide, allowing cable TV companies to have 
optimum arrangement in facing clients and video program providers, in 
order to maximize the number of programs while minimize the number of 
channels. 

It shall be known by those who are skilled in the art that the video 
compressor and the video compressing and restructuring device suggested 
by the present invention are not limited by the aforementioned embodiments. 
Their configuration or design^ as well as constituent number can be 
modified to adapt to system requirements, which are not beyond the scope 
of the present invention, 

A preferred embodiment of the hardware for the present invention is 
that a digital video compressing and restructuring device comprises a 
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mother board and 3 pieces of Single Board Personal Computer (SBPC); the 
mother board comprises Central Processing Unit (CPU), Read Only 
Memor}' (ROM), Random Access Memory ( RAM), disc drives, and high 
speed network interface; each SBPC comprises CPU, ROM, RAM; 8:1 
5 muitipiexe 650 can be made of a CPU (or a computer). 

A preferred embodiment of the trancoder is shown in Fig. 6(a) and 
Fig. 6(b). Fig. 6(a) briefly illustrates trancoder 700 comprising decoder 702 
for decoding inputted bit stream (digital video data stream) Bl and encoder 
704 for receiving the bit stream (digital video data stream) decoded by 

10 decoder 702 and encoding it into bit stream (digital video data stream) B2. 
Detailed description of trancoder 700 is shown in Fig. 6(b) where delay 
buffer 712 adjusts inputted bit stream (digital video data stream) Bl and 
generates an overflow signal according to its overflow status; quantization 
scale predictor 714 estimates, based on nonlinear algorithm, optimum 

15 quantization scale according to the current overflow status and the video 
signal segment to be outputted immediately: variable length decoder 716 
restores the signal produced by a variable length encoder to numeral codes; 
decRiautizer 718 restores quantized signal; quantizer 720 proceeds another 
tiuantization according to the outputs of quantization scale predictor 714 

20 and dequantizer 718; its output is processed by variable length encoder 722 
to pro\-ide bit stream (digital video data stream) B2 as an output. 

The trancoder is characterized in that the parts of video signal which 
are to be well sensed by human eyes are less compressed while those which 
are to be less sensed by human eyes are more compressed, in order to 

25 achie\ e maximum compression while maintain frame quality in the range 
human eyes can tolerate. 

The compression can be easily done by software in a personal 
computer for meeting most requirements for video display quality. The 
aigoritlim for compressing data in the present invention is to determine a 

30 new quantization scale when quantizing data, i.e., relatively rough 
quantization scale is given to the complicated parts (the parts with 
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, i j=l, ,8 (equation!) 



roughness not easily sensed by human eyes) of a frame, while relatively fine 
quantization scale is given to the plain parts (the parts with roughness easily 
sensed by human eyes) of a frame. 

In the operation of MPEG2 compression, image processing is done 
5 on the basis of basic unit (Macroblock; MB) which comprises 8X8 Pixels. 
The image signal contained in a MB is processed by a discrete cosine 
transformation to become a transformation coefficient C y ; quantization is 
one of several main steps in the MPEG compression of video sgnals. If 
transformation coefficient C y is divided by quantization step size, and then 
10 an operation of making integers is applied, quantization levels are 
obtained below 

where q^ is quantization scale, and is an integer ranging from I to 31 in 
MPEG2; 6i)ij is a quantization matrix for applying different weighting of 

15 processing to the transformation of different locations, the weighting is 
established through observation by human eyes; practically however, the 
higher frequency the transformation coefficient is associated with, the less 
sensitivity it has to human eyes, and the corresponding locations in the 
matrix have bigger coefficient (less fine), while the locations corresponding 

20 to a transformation coefficient associated with lower frequency have 
smaller matrix value which leads to finer quantization step size, here a is a 
quantization constant, and is assigned to equal 2^. 

Each video frame having a necessity of bit rate trancoding shall have 
its frame type remain unchanged, and have the number of its total bits and 

25 each average quantization scale as well as the number of corresponding bits 
recorded. Take I Frame for example, assume the number of bits of a 
temporarily recorded frame is B^^^^'bits, the bit rate of inputted video signals 
is Rj Mbps, and the bit rate of outputted video signals is R 2 Mbps, the 
desired number of bits (T bits) of trancoded output for the frame is obtained 
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according to the ratio between the bit rates as follows. 



T = ^-BP"" (equations) 



" ^^Mmber T is the desired number of bits set before the frame is 



1 lie lie 



irancoded, and is theoretically an ideal number of bits of the trancoded 
output for the frame. The object of controlling bit rate is to make the 
number of bits of the trancoded output for the frame approximate the 
desired number of bits. 
10 After calculating the desired number of bits fir a frame, the 

Complexity estimation Cj of each MB of the frame is then computed, and 
tlie desired number of bits (T^bits) of each MB is allocated according to the 
Complexity estimation Cj of the MB, as shown below, 

15 I-^, - C1 + C2 + + Cm^ ^ 1 ^i^^ (equation 4) 



C-qj?BP^^"j,j=L ,m (equation 5) 

20 \\ here m is the number of all MBs in the frame, T is the desired number of 
ail bits in the frame. Computation of C j is shown by equation 5 vAiere is 
the quantization scale of the j th MB of an inputted frame, B^'^j is the 
number of the bits wliich are in the inputted frame and are enclosed by the 
MB. Because the input to the trancoder is MPEG2 video signals, the 

25 encoded data for inputted video signals can be known when proceeding 
trancodiiig, and higher efficiency and accuracy can be thus achieved by 
setting desired number of bits according to the Complexity estimation Cj 
of each MB. 

Whenever the trancoding for a MB is completed during the process of 

13 



trancoding, the overflow coefficient of virtual buffer shall be updated as 
shown by equation 6 below, 

d^ = d*o + - {equation 6) 

5 where d'j is the overflow coefficient of virtual buffer when trancoding the j 
th row, B^Vi is the number of bits of the output for the (j-l) th row, Tj.i°* is 
the desired number of bits computed by equation 4 for the 0 -1) th row. 

It can be seen from equation 6 that d) is successively accumulated. 
In case the number (B"*^ ) of bits of the trancoded output for each row before 
10 the (j-l ) th row exceeds the computed desired number T"*^, dj' will gradually 
become bigger until Quantization scale gets so big that the number of 
outputted bits starts to be smaller than desired number of bits. This is tiie 
time the overflow coefficient begins to fall off. 

In equation 6, d^o is the initial value of overflow coefficient for I 
15 frame, the initial value in the beginning is 

d^o=q.eed* (equation 7) 

where 7 is the value obtained throu^ dividing bit rate by the number of 
frames per second, i.e., 



20 



bit rate , ^. 

Y^2 = (equation 8) 

frame _ rate 



qseed = ?i'exp 



P 



(equation 9) 



25 where ql is the quantization scale of the first MB of the the 

first frame, j8 is a coefficient related to ql and is used as the initial value of 
the overflow coefficient for next I frame. For P frame and B frame, the steps 
before computing overflow coefficient are the same as those for I frame. 
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For each MB, the quantization scale predictor suggested by the 
present invention can be used to obtain in advance the q,**^* (Optimal 
Quantization scale ) given that the current overflow coefficient d^.i and its 
desired number T"^i of bits are known. The predication based on d^.i and 
5 J-^^, is usually not good enougli, because the predication for best qi*^* 
based on current di_i and Ti""^ may heavily affect the qi+i°^^ for next MB, such 
as the case Tj+i^ becomes ver>^ large while dj is not big enough, resulting in 
a poor scale to quantize Ti""^ for qi+i'^\ Observation of more Tj'"^(j>l) 
will be more proper for determining relatively suitable qi^^*. It must also be 

10 noted that the relations between q^^^ and d^.i, T;"^, Ti+i°^, are nonlinear, 

and therefore the computation for the predication can be based only on 
experienced formula associated with complicated computation and 
accompanied with inaccuracy. It is therefore an object of the present 
invention to provide a neural network workable with learning approach in 

15 order to better define the relations between qi^*^ and d,.,, Ti^^ Ti^j^, 

Fig. 7 shows a preferred embodiment of a neural network which is 
a 3 layer of Multi-Layer Perceptron (MLP). It comprises an input layer 
802. a concealed layer 804, and an output layer 806. Try each of various 
different values for di.^, T^^j, V^^i+i, , to find, by human experimentation, 

20 a q^"^* for best fi*ame performance, and then train the neural network 
according to these values. Due to its Generalization capability the neural 
netvrork can make optimum predication for various cases. It must be noted 
that the output value of the neural network ranges between 0 and 1, thereby 
the outputted q{'^^ appears as a normalized value which must be multiplied 

25 by a constant. 

Fig. 8 shows an application example of the cable TV broadcasting 
and receiving system suggested by the present invention. Configured on 
broadcasting site are 8 digital video compressing and restructuring devices 
901-908 forming a single channel tlirough multiplexer 910, with video 

30 output fed to cable 916 through modulator 912 and frequency multiplier 914, 
for users to retrieve video programs fi*om set-top-box 918 on remote site 



and display the programs on TV set 920. The operation of set-top-box 918 
is the same as the set-top-box 300 shown in Fig. L 

The present invention' feature of enabling a single channel to 
accommodate many programs contributes significantly to the establishment 
5 of a Video On Demand (VOD) system. Fig.9 shows an analogy Video On 
Demand system (NVOD) provided by the present invention, in which a 
digital video compressing and restructuring device 930 as that shown in Fig. 
5 is configured on broadcasting site, and 24 video programs are merged into 
a single channel. There can be various options for the source of the video 

10 programs, among which are video tape 931, Compact Disc (CD) 932, 
compressed video signals, digital video disc (DVD) 933, and floppy disc 
934 containing compressed image, etc. After being integrated by digital 
video compressing and restructuring device 930, and broadcast through 
cable system 935 or through satellite antenna 936 as well as uplink satellite 

15 937, these programs can be directly received by users through satellite 
antenna 938, or received by cable TV service companies through satellite 
antenna 939 and then fed to cable system 935 via headend 940. Because 
24 programs can be merged in a channel, if a hot program is broadcast 
through a sub-channel every 2.5 minutes, by considering 2.5 minutes X 

20 24=60 minutes (equation 10), 

it can be seen that the broadcasting of a movie based on a NVOD provided 
by the present invention can proceed with original video signals of one 
copy 

While the invention is described in terms of what are presently 
25 considered to be the most practical and preferred embodiments, it must be 
understood that the invention is not limited to the disclosed embodiment. 
On the contrary, it is to cover various modifications and similar 
arrangements included within the spirit and scope of the following claims 
which are to be accorded with the broadest interpretation to encompass all 
30 modifications and similar structures based thereon. 
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